Is It Enough to Get the Behavior Right?

This paper deals with the relationship between intelligent behaviour, on the one hand, and the mental qualities needed to produce it, on the other. We consider two well-known opposing positions on this issue: one due to Alan Turing and one due to John Searle (via the Chinese Room). In particular, we argue against Searle, showing that his answer to the so-called System Reply does not work. The argument takes a novel form: we shift the debate to a different and more plausible room where the required conversational behaviour is much easier to characterize and to analyze. Despite being much simpler than the Chinese Room, we show that the behaviour there is still complex enough that it cannot be produced without appropriate mental qualities. In this paper, we will consider the issue of the relationship between external behaviours and mental qualities. The external behaviours we have in mind are the linguistic responses in an intelligent conversation. The mental qualities we have in mind are things like knowing how to speak a language, or understanding what is being said, or even being intelligent (none of which we will need to distinguish for now). The fundamental issue we intend to investigate is this: When can we justifiably draw conclusions about mental qualities like these, given external behaviour that is indistinguishable from that of a person? In a sense, this question is not really part of AI. One definition of AI is that it is “the study of intelligent behaviour achieved through computational means” [Brachman and Levesque, 2004]. From this point of view, AI research is about getting the behaviour right and nothing more. The question above goes beyond this and asks what conclusions we can draw should we ever get the behaviour right. The Turing Test [Turing, 1950] and the Chinese Room [Searle, 1980] are two thought experiments designed to help us understand this issue. To recap the positions very briefly, we have Turing who says (roughly) that the mental vocabulary above is too vague and open to interpretation to be worth arguing about. If we are unable to distinguish the responses of an entity from that of a person in an unrestricted conversation (in what Turing calls the Imitation Game), that ought to be enough. In short: if the behaviour in the long run is what it should be, we should be prepared to ascribe the same mental qualities we would to a person. Searle, on the other hand, imagines himself in a room called the Chinese Room where there is a large book. People give him messages written in Chinese, which he does not understand. However, the book in the room tells him what to do with this message, culminating in him writing characters on a piece of paper, the meaning of which he does not understand. He hands the paper back outside the room, and the people there find these responses quite congenial, and in fact indistinguishable from those of a native Chinese speaker. But Searle does not know Chinese. He is producing linguistic behaviour that is indistinguishable from a native speaker’s without any understanding. So getting the behaviour right does not justify the ascription of the mental qualities. So Turing and Searle take opposite positions on the issue above. But one aspect that they both would agree on (one imagines) is this: when we talk about getting the behaviour right, and in a way that is indistinguishable from someone with the appropriate mental qualities, we are not talking about doing so in some limited context. All parties agree (or would likely agree) that it is possible to use trickery and other uninteresting means to get the behaviour right in conversations that are restricted enough. For example, a conversant that says nothing but “I love the Yankees!” over and over might be producing conversation that is indistinguishable from that of fanatical baseball fan (possessing mental abilities beyond the four words, one still presumes), but nothing interesting follows from this. Similarly, a test that is limited in advance to a certain number of words may not be enough. What matters to both Searle and Turing and what concerns us here are the conclusions that we would draw about the mental properties of a conversant given that the conversation is natural, cooperative, unrestricted, and as long as necessary. 1 The AI perspective and the Systems Reply So who is correct here, Turing or Searle? Much of the debate within AI has not really attempted to answer the question. Regarding the Turing Test, the main discussion has been on whether linguistic behaviour is enough, or whether a more comprehensive notion of behaviour would be a better test [Harnad, 1989]. For example, we might want a notion of behaviour that encompasses broader notions of perception